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^^Quality education is the most valuable asset 
for present and future generations. Achieving 
it requires a strong commitment from everyone, 
including governments, teachers, parents and 
students themselves. The OECD is contributing 
to this goal through PISA, which monitors 
results in education within an agreed framework, allowing 
for valid international comparisons. By showing that some 
countries succeed in providing both high quality and equitable 
learning outcomes, PISA sets ambitious goals for others. 




Angel Gurria 

OECD Secretary-General 



Building on PISA’S Knowledge Base 



D eveloped jointly by OECD member countries through the 
OECD’s Directorate for Education, the Programme for 
International Student Assessment (PISA) aims to measure 
how far students approaching the end of compulsory education 
have acquired some of the knowledge and skills essential for 
full participation in the knowledge society PISA is an important 
part of the work of the Directorate for Education, which collects 
data and provides comparative indicators of education systems 
in OECD member and partner countries. 

In 2006, PISA completed its first set of surveys of the knowledge 
and skills of 15-year-olds in OECD member countries and partner 
countries, covering 90% of the world economy These PISA surveys 
examined the performance of students in key subject areas and 
also looked at a wider range of educational outcomes that 
include students’ motivation to learn, their beliefs about 
themselves and their learning strategies. They examined how 
performance varies between the genders and between socio- 
economic groups. They also provided insights into some of the 
factors that influence the development of knowledge and skills 
at home and at school, how these factors interact and what the 
implications are for policy development. Most importantly, PISA 
has shed light on those countries that succeed in achieving high 
performance standards while at the same time providing an 
equitable distribution of learning opportunities. The first three 



PISA surveys each focused on a particular subject area: reading 
(in 2000), mathematics (in 2003) and science (in 2006). The 
programme will now conduct a second set of surveys in 2009, 
2012 and 2015. 

In addition to monitoring student performance in the three 
main subject areas, PISA will seek to deepen policy insights by: 

> Developing better ways of tracking student progress, including 
exploring the possibility of comparing student progress 
between primary education and the age of 15 years 

> Allowing closer comparisons between performance and 
instruction, by extending the option of examining student 
performance within a single grade and gathering more data 
about classroom experiences 

> Making use of computer-based assessments, not only to 
measure Information and Communication Technology (ICT) 
literacy skills but also to allow for a wider range of dynamic 
and interactive tasks and to explore more efficient ways of 
carrying out the main tests of student knowledge and skills 
in reading, mathematics and science 

These innovations will initially be explored by countries as a 
supplementary, optional component of PISA, but where appropriate 
may be integrated into the survey’s core. 
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The Story So Far: 

P ISA seeks to measure how well young adults, at age 15 
and therefore approaching the end of compulsory schooling, 
are prepared to meet the challenges of today's knowledge 
societies - what PISA refers to as “literacy”. The assessment is 
forward looking, focusing on young people's ability to use their 
knowledge and skills to meet real-life challenges, rather than 
merely on the extent to which they have mastered a specific 
school curriculum. This orientation reflects a change in the 
goals and objectives of curricula themselves, which increasingly 
address what students can do with what they learn at school 
and not merely whether they can reproduce what they have 
learned. 

Well over one million students have so far been assessed. In 
addition to paper-and-pencil tests measuring reading, mathematical 
and scientific literacy, students filled in questionnaires about 
themselves, while their principals completed questionnaires 
about their schools. This has produced an unprecedented 
comparative knowledge base of school systems and their outcomes, 
and allows these outcomes to be monitored over time. 



DID YOU KNOW 



that more than 60 OECD member 
countries and partner countries have participated in PISA so far 
and that they cover roughly 90% of the world economy? 



PISA 2000-2006 
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PISA’S key features 

> Its policy orientation, with design and reporting methods 
determined by the need of governments to draw policy lessons 

> Its innovative “literacy” concept, which is concerned with the 
capacity of students to apply knowledge and skills in key 
subject areas and to analyse, reason and communicate 
effectively as they pose, solve and interpret problems in a 
variety of situations 

> Its relevance to lifelong learning, which does not limit PISA 
to assessing students’ curricular and cross-curricular competencies 
but also asks them to report on their own motivation to learn, 
their beliefs about themselves and their learning strategies 




The Story So Far: PISA 2000-2006 




> Its regularity, which will enable countries to monitor their 
progress in meeting key learning objectives 

> Its contextualisation within the system of OECD education 
indicators, which examine the quality of learning outcomes, 
the policy levers and contextual factors that shape these 
outcomes, and the broader private and social returns to 
investments in education 

> Its breadth of geographical coverage and collaborative nature, 
with more than 60 countries (covering roughly nine-tenths 
of the world economy) having participated in PISA assessments 
to date, including all 30 OECD countries 




Development of PISA 

In its first three surveys, PISA has been characterised by 

> The establishment of comprehensive 
frameworks for assessing readingp 
mathematics and science 

> In each three-yearly PISA survey, one subject was chosen as 
a focus while two other subject areas have been assessed 
more briefly This allowed, for each subject area, a detailed 
profile of what a country’s students can do every nine years, 
and an update of their performance every three years. The 
assessment framework for each subject was finalised in the 
year that it featured as a focus. The development of these 
frameworks has been central to PISA, which takes an innovative 
approach to assessing knowledge, skills and attitudes, by 
identifying the key concepts and processes that underpin 
each subject, rather than merely examining curriculum content. 




> Exploration of student abilities and characteristics 
beyond school subjects 

> From its outset, PISA has assessed student performance beyond 
the confines of reading, mathematics and science, as cross- 
curricular competencies like ICT skills, communication and 
problem-solving will be important to students’ futures. The 
surveys asked students about their characteristics as learners, 
including their learning habits, their motivation levels, their 
attitudes to learning in general and to reading, science and 
mathematics in particular, and their habits and competence 
in using ICT. This has allowed analysis of student approaches 
to learning and of self-assessed cross-curricular competencies 
such as computer proficiency. In addition, the direct assessment 
in 2003 of student performance in solving problems, outside 
the context of a single curriculum area, was a first step in 
describing such competencies through external testing rather 
than self-reporting. 



DID YOU KNOW 



...that each PISA cycle has produced 
more than 10 OECD publications and numerous national 
reports in the participating countries? 





> Analysis of studentSp schools and school systems' 
characteristicsp set alongside performance data 

> PISA’s database provides an unprecedented array of information, 
based on questionnaires, from which to analyse potential 
influences on student performance across countries. Although 
it does not track individual students and cannot therefore 
establish causal links, this analysis can compare the degree 
of association with educational outcomes of various factors 
in different countries. At the individual level, such factors 
include socio-economic background, immigration status and 
cultural possessions in the home. At the school level, they 
include student perceptions of instructional practices, 
disciplinary environment and, importantly, the collective 
socio-economic background of students at each school. At 
the school system level, the extent of school autonomy and 
the structural organisation of students in secondary education 
can be compared to the overall performance and distribution 
of the performance of students aged 15. 



The Story So Far: PISA 2000-2006 
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Taking Forward a Global 
Collaboration to Monitor 
Education Systems 

P ISA is a collaborative effort, bringing together scientific 
expertise from the participating countries, steered jointly 
by their governments on the basis of shared, policy-driven 
interests. The project’s Governing Board, on which all participating 
countries are represented, decided in 2005 to take forward this 
partnership into a new phase lasting from 2009 to 2015. As in 
the first phase, PISA will aim to better assist countries in 
understanding the processes that shape quality and equity in 
learning outcomes within the educational, social and cultural 
contexts in which education systems operate. 






Taking Forward a Global Collaboration to Monitor Education Systems 
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Maintaining the knowledge base: 
PISA 2009-2015 



The central part of the PISA survey will continue to be a two- 
hour assessment of 15-year-olds’ knowledge and skills, focused 
on reading, mathematics and science, combined with questionnaires 
filled out by students and school principals. However, this core 
activity will continue to be strengthened and developed, with 
the potential for adding new elements, either permanently or 
as one-off components. Specifically, the core of PISA will: 

> Remain focused on assessing the cumulative yield of education 
at the age of 15 years, the highest age at which participation 
in formal education in OECD countries is still largely universal. 
This will allow the impact of educational change to be tracked 
and compared across countries for at least the first 15 years 
of this century. 

> Continue to assess the extent to which students have acquired 
key competencies and dispositions needed for further learning 
and for adult life, and therefore focus on proficiency in applying 
knowledge acquired at school to a wider context, rather than 
in just reproducing it. 

> Build on existing assessment frameworks in order to improve 
their efficiency in assessing desired outcomes, but without 
compromising comparability of results over time. 



> Include in each survey one developmental assessment area 
to be selected by participating countries. This could be a 
component involving test performance, like the assessment 
of problem-solving skills in 2003, or it could rely on student 
self-assessments, as with self-reported attitudes to science 
in 2006. 

> Continue to contextualise performance by administering 
questionnaires to students and principals, and extend the 
analytical power of PISA by adding an optional parent 
questionnaire. 

> Provide the opportunity for countries to further describe 
students’ instructional settings by linking PISA to other surveys, 
e.g. teacher surveys. 







Extending our understanding: three new areas of enquiry 



W ith the core programme of work, PISA places the 
emphasis on cross-national comparisons of the 
cumulative yield of education towards the end of 
compulsory education. It assesses to what extent education 
systems succeed in ensuring that young adults acquire the key 
competencies and dispositions to learning believed to contribute 
to the foundations for further learning and a successful transition 
into adult life; as well as what individual, systemic and contextual 
factors contribute to this success. Because of the ongoing nature 
of PISA, the core programme will also provide trend indicators 
that will allow countries to measure improvements in educational 
outcomes. 




While the core of PISA allows countries to maintain a common 
and consistent approach to assessing student knowledge and 
skills over time, PISA seeks to remain open to methodological 
and substantive innovation. To this end, PISA will explore the 
development of optional modules which seek to extend policy 
insights into: 

> The measurement of learning progress over time and the 
comparison of progress across countries 

> The relationship between aspects of instruction and learning 
outcomes 

> The assessment of ICT competencies, as well as the use of 
technology as a means to capture a broader range of assessment 
tasks 



DID YOU KNOW 



...that the PISA survey is typically 
administered to between 4 500 and 50 000 students in each 
country.? And that these students are selected from a random 
sample of both public and private schools and are chosen 
according to age (from 15 years and 3 months to 16 years 
and 2 months at the beginning of the assessment) 
as opposed to which grade they belong? 





Taking Forward a Global Collaboration to Monitor Education Systems 
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> Assessment of student progress: How can we best measure the progress of students in acquiring 
knowledge and skills in the course of their education? 



The objective of this optional module would be to investigate 
the possibility of measuring learning progress within education 
systems, by linking assessments of student knowledge and 
skills at different stages of schooling. 

PISA has so far given a snapshot of student performance at a 
single point of time, near the end of compulsory education. 
While this gives an indication of the cumulative yield of school 
systems, it does not show the extent to which students progress 
over time. This requires information on performance at different 
stages of education. 

Although some countries have programmes of tracking the 
individual students assessed in PISA, it is not at present feasible 
for all the countries to do this. However, measuring the overall 
performance of students at different ages in different education 
systems does allow comparison across countries of the progress 
of whole student populations. Ideally, periodic surveys at different 
ages would be designed to ensure that a given cohort’s performance 
is measured at different ages in successive surveys. 



The integration of the assessment of a younger age cohort would, 
in particular: 

> Give an indicator of outcomes of the earlier years of education, 
and allow analysis of whether they are sustained through 
secondary school 

> Help link PISA outcomes more closely to what happens within 
schools and classrooms: a connection that is easier to measure 
in the more uniform organisation of primary schools, where 
each student generally has one main teacher 

> Be able to look at how factors other than cognitive performance 
develop over time - for example, whether students’ levels of 
motivation change and how their learning styles evolve 

> Allow a common framework to be developed for monitoring 
the development of certain basic skills, which could be 
translated into estimates of the average yearly progress made 
by students in subjects such as reading 

This option is a longer-term strategic objective for PISA but 
there are no plans for its implementation in the context of the 
PISA 2009 assessment. 





> Instruction and outcomes: How can we improve our capacity to describe those aspects of 
instruction that influence student outcomes? 



The second optional module would facilitate analyses of 
instructional aspects that could lead to enhanced learning 
outcomes. 

PISA is currently able to show the extent to which positive 
learning outcomes are associated with particular features of 
schooling. So far, the strongest effects have been noted at a 
fairly general level. For example, students in schools with a 
strong disciplinary climate do better on average. More precise 
links with the educational experiences of individuals have been 
limited by the scope of questionnaires administered and by the 
fact that 15-years-olds are not all in one grade or one class, 
making it hard to draw precise links with their experiences in 
the classroom. 

This optional module seeks to enable countries to trial several 
extra components in the 2009 survey in order to permit better 
measurement of the links between students’ instructional 
experiences and their performance. These will all be developed 
further in subsequent surveys. 



This module would comprise three separate components: 

> A supplementary assessment of students in a single grade or 
class (in addition to the usual survey of a sample of 15-year- 
olds), allowing the characteristics of students’ instructional 
experience to be described more precisely 

> A more comprehensive collection of information about students, 
schools and school systems, based on extra questions posed 
to students and principals, and a collection of more data about 
school systems as a whole 

> The exploration of links with the OECD’s Teaching and Learning 
International Survey (TALIS), which will first take place in 
2007 and which will enable extensive analysis of differences 
in national approaches to teaching and learning 
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> Computers and PISA: How can ICT strengthen the assessment of students' knowledge and skills? 



The third optional module would seek to develop computer- 
delivered assessments to facilitate better coverage of those 
aspects of the PISA frameworks that are very difficult to capture 
with static paper-and-pencil documents. In this way, the 
assessment process will become more efficient (e.g. via adaptive 
testing), with reductions both in operational costs and in student 
response time. In the longer term, these developments should 
improve the assessment’s targeting across and within countries. 

ICT forms an essential part of life in the modern world. Besides 
the computer’s relevance for student assessment, ICT competencies 



in themselves are also becoming a highly relevant subject for 
assessment. 

In its first phase, PISA made a start in both areas. In 2003, it 
used a student questionnaire to survey the extent to which 
students used computers and felt comfortable using them. In 
2005, 13 countries undertook a pilot computer-based assessment 
of science. The second phase will take this work further, and 
potentially lay the groundwork for introducing a computer- 
based assessment in the PISA core. 



> Anticipated timeline for the computer-delivered assessment and the assessment of ICT literacy 




Development 

Activity 



Assessment 

(see comments 
on previous 
diagram) 



Test whether computerised 
assessment is compatible 
with penciL-and-paper tests; 
develop framework for 
assessing ICT literacy. 



Assessment of reading electronic 
texts 



Develop “adaptive” assessment 
(see below) 



Implement supplementary computer 
assessment in focus areas 
Assess ICT skills 

Pilot “adaptive” technology enabling 
students of different abilities to 
respond to different sets of questions 




Continue implementation of all parts 








U p to and including the 2009 survey, this work will focus 
on assessing the reading of electronic texts. It will only 
be possible to explore fuller assessments in each of the 
three subject areas and of students’ ICT skills for the 2012 survey 
and onwards. This reflects the importance of developing 
instruments that are fully compatible with the rest of PISA, 
especially when using computers to assess reading, mathematics 
and science. 




Over the longer term, there is potential for computer-based 
assessments to play a part in improving the delivery of the PISA 
core, bringing several advantages. In particular: 

> It may be possible to assess certain types of thinking in 
computer-based assessments that are difficult to assess in 
pencil-and-paper tests. A pilot in 2005 showed, for example, 
that it is possible to simulate the sequential steps taken in 
scientific experiments by this method. 

> A computer-delivered approach could, in the longer term, 
ensure that tests align better with the performance levels of 
individual students, giving them more challenging or more 
manageable tasks according to how they perform in early 
questions. This permits in particular a more fine-grained 
analysis of the most and least able students. It also provides 
a more useful profile of countries whose students are clustered 
in particularly high or low areas of performance. 

> Finally, computer delivery brings practical advantages, including 
lower data entry costs. 




PISA’S Future: Monitoring Change, Deepening Understanding 
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PISA’S Future: Monitoring Change, 
Deepening Understanding 



I n 2009, the students who will be assessed in PISA will have 
entered primary school at about the time of the first PISA 
survey in 2000. As a result, countries will be able to gauge to 
what extent any changes in their education systems may have 
produced improved outcomes compared with the benchmarks 
set by the original 2000 survey. A central objective of PISA - 
providing a stable point of reference against which to monitor 
the evolution of education systems - will therefore have been 
achieved. 

At the same time, PISA continues to seek a deepened understanding 
of strong student performances, to get better at measuring them 
and to examine links between outcomes and educational 
processes. In the first set of surveys from 2000 to 2006, preliminary 
steps were taken to measure a wider set of competencies than 
had been tested in international surveys, and to collect contextual 
data that would show which factors are linked to student 
performance. The new elements of PISA described here will take 
this process further, without compromising PlSA’s integrity as 
a stable instrument for monitoring student knowledge and skills. 



Above all, this evolution is guided by the priorities of educational 
policy makers, who want to ensure that further changes in 
education systems are firmly rooted in good evidence. In the 
coming years, PISA will offer countries more elaborate instmments 
for making linkages between students’ experiences at school 
and their knowledge and skills near the end of secondary 
education. By taking the developmental approach to innovations 
in student assessment described above, PISA will be able to test 
which methods work well enough to eventually be incorporated 
into its core. 






DID YOU KNOW 



...that you can try sample interactive questions 
from PISA and consult all publicly released test questions on the 
PISA websiteF 



...that the micro-level PISA data, as well as 
their accompanying manuals, are available for further analysis 
on the PISA websiteF 



www.pisa.oecd.org 

www.oecd.org/media 

www.oecd.org/publications 

www.oecd.org/statistics 




ORGANISATION FOR ECONOMIC 
CO-OPERATION AND DEVELOPMENT (OECD) 

The OECD groups 30 member countries sharing a commitment 
to democratic government and the market economy and provides 
a unique forum for them to discuss, develop and refine economic 
and social policies. The OECD’s mission is to promote policies 
designed to: 

> Achieve sustainable economic growth and employment and 
rising standards of living in member countries while maintaining 
financial stability, so contributing to the development of the 
world economy 

> Assist sound economic expansion in member countries and 
other countries in the process of economic development 

> Contribute to growth in world trade on a multilateral, non- 
discriminatory basis 

The OECD also has active relationships with some 70 other 
countries, non- government organisations and civil society, and 
these help give its activities a global reach. 



DIRECTORATE FOR EDUCATION 

The OECD views education in a “cradle-to- grave” lifelong learning 
framework, involving formal, informal and non-formal settings. 
Internationally comparable statistics and indicators underpin 
the work but it has a strong qualitative dimension as well. The 
ultimate outputs are policy recommendations designed to 
increase both the quality and equity of education systems. The 
OECD’s work on education is relevant, not only for government 
and local authorities, but also for civil society, researchers, 
professional practitioners and an informed lay audience. The 
extent of the audience is evident in the public discussion and 
debate that OECD’s publications on education often generate. 
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